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(54) System and method for displaying of colour images. 



® 



A system and method are provided for utilizing, in a colour imaging environment, such as ray tracing, 
display adapter hardware which is capable of interpolating colour values for pixels lying in an area 
between previously colour imaged pixels. A computer system CPU performs ray tracing calculations, or 
the like for a predetermined number of the total pixels, rather then each pixel, as is the case with 
conventional ray trace image systems. The system CPU then turns over the colour imaged pixels to a 
display adapter, capable of colour interpolating therebetween. In this way, previously idle display 
adapter hardware can be utilized to determine pixel colour values and allow the system CPU to continue 
colour imaging other more critical areas of a scene to be displayed. Therefore, an image can be 
rendered at a much faster rate than possible with conventional systems, with relatively no visible loss in 
quality. 
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SYSTEM AND METHOD FOR DISPLAYING OF COLOUR IMAGES 



The present invention relates generally to the 
area of computer graphics and more particularly to 
methods and means for displaying of colour images 
on display hardware. There are at least two distinct s 
methods of placing colour values (r ( g.b) into the single 
picture elements (pixels) contained on a hardware 
screen : "rendering of geometry, " and "displaying of 
image." "Rendering of geometry'' is a method in which 
objects to be displayed on the screen are expressed 10 
as polygons. Vertices of these polygons have 
associated colour values determined by calculations 
(lighting operations) performed at each rendering. 
Pixels, chosen at each rendering, are given these col- 
our values to display the vertices. Remaining pixels is 
are given colour values via "shading". "Shading" is the 
process of filling in the rest of the pixels that comprise 
a polygon during "rendering of geometry." Color 
values for these pixels are calculated from information 
known about the vertices of the polygon. "Gouraud 20 
shading" is one method of shading in which the colour 
values of the vertices are colour interpolated to derive 
the colour values for the remaining pixels of the poly- 
gon. There are other methods of shading which do not 
involve colour interpolation. In contrast to "rendering 25 
of geometry," pixels may be given colour values by 
simply filling them in from an array of colour values 
called an "image." In this case there are no 
established objects or polygons expressed to the dis- 
play hardware. Such an image may have been 30 
obtained from a camera, or from a process of calculat- 
ing colour values such as "ray tracing-" "Ray tracing" 
is a computation intensive method of calculating col- 
our values for a pixel. Typically, a computer system 
central processing unit (CPU) is utilized to perform 35 
these imaging computations. In these conventional 
systems, the CPU performs all of the imaging (ray 
tracing) computations for each and every pixel on a 
display screen in sequential order. For a detailed dis- 
cussion of ray tracing, see "An Overview of Ray Trac- 40 
ing," Andrew S. Glassner. It can be seen that imaging 
computations in general and ray tracing in particular 
are extremely time consuming and may take a matter 
of hours, or even days to complete this colour imag- 
ing, depending upon the description of the screen 45 
being displayed. The combination of calculating and 
displaying an image, may be called "colour imaging." 

Generally, a number of currently available com- 
puter display adapter hardware devices are capable 
of performing rendering of geometry, including so 
Gouraud shading. Gouraud shading is a graphics 
technique unrelated to colour imaging, or ray tracing. 
Since Gouraud shading is implemented through col- 
our interpolation, these display adapters are capable 
of performing colour interpolation. The present inven- 55 
tion utilizes this colour interpolation capability to 



achieve a goal not contemplated by the conventional 
and standard uses of Gouraud shading. 

During the period when a CPU is performing the 
ray tracing calculation, all other associated computer 
graphics hardware remains idle, awaiting completion 
of these computations. Therefore, it would be ext- 
remely desirable to provide a method which would 
optimize the efficiency of a computer graphics system 
by utilizing the capabilities of this idle graphics 
hardware in conjunction with the processing being 
conducted by the system CPU. 

Viewed from a first aspect, the present invention 
provides a method of displaying a colour image by 
interpolating pixel colour values, said method com- 
prising the steps of : 

determining a number of pixels to be colour 
imaged ; 

computing colour imaged values for each of 
said pixels included within the determined number ; 
and 

using the colour imaged values to interpolate 
colour values for each pixel not included within the 
determined number. 

Viewed from a second aspect, the present inven- 
tion provides a system for displaying a colour image 
by interpolating pixel colour values, comprising : 

interface means (22) responsive to user input 
to determine a number of pixels to be colour imaged; 

a processing unit (18) to compute' colour 
imaged values for each of said pixels included within 
the determined number ; and 

interpolation means (12) for using the colour 
imaged values to interpolate colour values for each 
pixel not included within the determined number. 

In contrast to the prior art, the present invention 
allows a microprocessor included within an 
associated display adapter to be used to alleviate a 
portion of the computational burden imposed upon the 
system CPU during colour imaging calculations. The 
present invention utilizes the colour interpolation 
capability of associated display hardware to increase 
the speed of a computer graphics system performing 
colour imaging. 

Initially, the system CPU performs colour imaging 
(e.g. ray tracing) on a predetermined plurality of 
pixels. At this point, the CPU then transmits the (r.g.b) 
values for each of these pixels to the processor of the 
display adapter hardware. The transmission (i.e. com- 
munication between the CPU and display adapter) to 
the hardware, of these colour values is in terms of a 
polygon to be Gouraud shaded, even though "render- 
ing of geometry" is not the method being used. Upon 
receipt of these pixel colour values, the display adap- 
ter processor then performs colour interpolation for all 
of the pixels included within a geometric figure 
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bounded by the colour imaged pixels. Thus, it can be 
seen that the number of calculations required to be 
performed for a given screen description is vastly 
reduced. This reduction in calculations greatly 
improves the speed with which the scene can be col- 
our imaged. 

The present invention will be described further, by 
way of example only, with reference to an embodi- 
ment thereof as illustrated in the accompanying draw- 
ings, in which : 

Figure 1 is a block diagram depicting the ele- 
ments utilized by a preferred embodiment of the 
present invention ; 

Figure 2, is a flowchart showing the steps 
required by the preferred embodiment to imple- 
ment the capabilities of the display adapter 
hardware ; and 

Figure 3 is a diagram of a portion of a display 
showing those pixels which may be colour inter- 
polated and those being colour imaged by a sys- 
tem utilizing the preferred embodiment. 
Typically, a computer graphics user that wishes 
to perform lighting calculations will utilize a three- 
dimensional (3D) display adapter hardware device 
16, such as a 2781 (High-Performance 3D Color 
Graphics Processor), sold by IBM. Included within the 
3D lighting hardware is a portion which is dedicated 
to performing shading operations. Usually, this shad- 
ing hardware supports Gouraud shading, as discus- 
sed above, however other shading hardware utilizing 
colour interpolation methods are contemplated by the 
scope of the present invention. During a normal light- 
ing operation, the shading hardware is utilized as a 
hardware assist to aid the system CPU in achieving 
the desired lighting of a displayed image. The present 
invention utilizes this hardware assist capability dur- 
ing non-lighting calculations, in particular during col- 
our imaging, such as ray tracing operations. 

Referring to Figure 1 , a block diagram of a system 
capable of utilizing the present invention is shown. A 
display 1 0, such as a CRT, or the like is shown having 
a plurality of picture elements (pixels) which are 
depicted by reference numerals 1, 2. 3, 4, 5, 6, 7, 8, 
9. 11, 13 and N. A display adapter 16 typically used 
for performing rendering of geometry is illustrated and 
includes a shading processor 12 and all other 
associated hardware 14. This hardware may include 
buffers, such as a frame buffer, window buffer, Z- buf- 
fer or the like. A system CPU 1 8 is provided which per- 
forms all colour imaging, such as ray tracing 
calculations. Also shown in Figure 1 is the system 
main memory 20. It should be noted that the present 
invention is suited for use in all types of colour imaging 
applications, however ray tracing is the primary 
method discussed herein. 

Conventionally, an application program interface 
(API) 22, via bus 26, instructs the 3D lighting display 
adapter 16 to perform rendering of geometry. This 



may include determining the location and consequ- 
ences of a light source, as well as colour interpolating 
5 a surface for shading purposes. It should be noted 
that in conventional systems, no colour imaging is 
performed by CPU 18 during rendering of geometry. 
Similarly, no rendering of geometry can be performed 
by prior art systems during the time colour imaging 
10 computations are being implemented by CPU 18. 

To perform colour imaging in a ray tracing envi- 
ronment a scene description is first stored in main 
memory 20. This scene description is a textual, or 
alphanumeric representation of the scene to be dis- 
is played. For example, a scene description will include 
geometric descriptions of all objects as well as their 
colour (r,g,b) values and surface type. Additionally, 
the nature of each object, is included within the scene 
description, that is, whether the object is transparent, 
20 opaque, reflective or the like. 

Upon implementation of a ray tracing operation, 
a program application user, via API 22 and bus 23, 
instructs the system CPU 18 to perform a series of 
vector (ray) calculations for each pixel on display 10. 
25 System CPU 18 begins at pixel 1 and back traces a 
ray from a view point directly in front of that pixel, 
through the pixel in the image plane, and into the 
scene. For example, if a ray 1 (corresponding to pixel 
1) is found to initially intersect the surface of the 
30 ocean, then a lighting calculation is computed at the 
point of intersection between ray 1 and the surface 
and CPU 18 colours pixel 1 the computed shade. In 
CPU 18. Furthermore, rf at pixel N the corresponding 
scene description is a convex reflective object, such 
35 as a mirrored sphere, then the ray tracing calculation 
includes back tracing along a first ray from the view 
point through pixel N to the point of intersection with 
the reflective surface, calculating the angle of reflec- 
tion to create a reflection ray which is then traced to 
40 find its nearest intersection with an object. These ref- 
lection rays are created and traced until an intersec- 
tion is found with a non-reflective object, at which 
point a shadow ray is created which is a ray from the 
point of intersection to the light source. If any interven- 
es ing intersections are found along this ray, i.e. interse- 
ction with other objects, then the primary intersection 
point is found to be in a shadow and the lighting cal- 
culation performed by the CPU 18 is adjusted accord- 
ingly. Upon completion of the lighting calculation at 
so this point of intersection, pixel N is coloured with the 
calculated shade. 

Thus, it can be seen how a number of reflective 
objects in the scene description can create an ext- 
remely time consuming and burdensome number of 
55 calculations to be performed by CPU 18. It is not 
uncommon for a period of hours or even days to 
elapse before a scene description is totally displayed 
on a CRT 10. This time lapse can cause a enormous 
burden on users of computer graphics systems. For 
example, a user of a computer graphics animation 
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system may have to wait all day for the scene contem- 
plated to be actually displayed such that it can be 
determined whether the scene description needs to 5 
be altered. It shouid be noted that display adapter 16 
represents the previously described lighting hardware 
and processor 12, the shading hardware portion of 
display adapter 16. 

Upon completion of the ray tracing computation, 10 
the CPU 18 then transmits, via bus 24, the calculated 
colour for that pixel to the display 1 0. Again, during the 
ray tracing calculation, display adapter 16, including 
shading processor 1 2 remains idle. 

In order to increase rendering speed, or more effi- 15 
ciently perform colour imaging, a preferred embodi- 
ment of the present invention utilizes shading 
processor 12 to colour a percentage of an image's 
pixels through colour interpolations of a subset of the 
ray traced pixels. 20 

A number of display adapters, such as the 
aforementioned 3D lighting hardware 16, are capable 
of filling (colouring) polygons onto display 10. Under 
typical colour image operating conditions CPU 18 
resolves individual pixels, as specified by a program 25 
application and then turns these pixels directly over to 
the CRT 10 for display. However, adapter 16, through 
shading processor 12, is capable of colouring poly- 
gons by interpolation methods, such as Gouraud 
shading, or the like. Generally, these polygons are 30 
simple polygons, such as squares, rectangles, 
triangles, and trapezoids. Therefore, the present 
invention exploits this capability of the display adapter 
hardware 16 to colour interpolate simple polygons. 

Specifically, a user of the computer graphics sys- 35 
tern of Figure 1 determines the number (or percent- 
age) of pixels to be colour imaged, or ray traced, prior 
to the colour interpolation being performed therebet- 
ween. For example, a user may desire that every 
other (one out of two) pixels be ray traced, such that 40 
pixels 1, 3, 5, and 7 are subjected to ray tracing cal- 
culations in sequential order. Upon completion of 
these four computations by CPU 18. their respective 
colour values are passed to the shading processor 12, 
via bus 28, of display adapter 16. In this example 45 
pixels 1, 3, 5 and 7 are the boundaries of a square, 
which is a polygon the display adapter 16 is capable 
of filling. Thus the CPU 18 is now free to continue ray 
tracing for another portion of the scene description, 
such as that portion bounded by pixels 3, 5, 1 1 and 13. 50 
Meanwhile, shading processor 12 of display adapter 
16 has interpolated the colour of pixels 2, 4, 6, 8 and 
9 from the known ray traced colours of pixels 1, 3, 5 
and 7. Thus, it is apparent how the present invention 
greatly increases the speed with which a ray traced 55 
image in a graphics system is rendered. That is, of 
pixels, 1 through 9, shown in Figure 1, only four pixels 
1, 3, 5 and 7 are actually ray traced, whereas five 
pixels 2, 3, 6, 8 and 9 are coloured by another pro- 
cessor. It should be noted that a user may choose the 



number and configuration of pixels to be ray traced. 
At one extreme, the four corner pixels of a display 10 
may be ray traced with the remaining pixels of the 
screen being colour interpolated. The other extreme 
may provide for ray tracing a high percentage of the 
total pixels, along the rows and columns of pixels on 
display 1 0 (see Fig. 3). By ray tracing one out of every 
2 pixels, only 25 percent of the total pixels are even- 
tually subject to ray tracing operations. Ray tracing 
every other pixel, as discussed above, has greatly 
increased rendering speed with an extremely minor 
deterioration in resolution. This minor deterioration in 
resolution is far outweighed by the increased render- 
ing speed. 

Figure 2 is a flowchart showing the steps required 
by the preferred embodiment to invoke and utilize the 
capability of the shading processor 12. 

At step 1, the hardware utilization method is 
initiated by a user of a computer graphics system. 
Next the user, or perhaps an associated program 
application, will determine a percentage of pixels 
which are to be ray traced. This percentage may 
range from a minute portion of the total pixels, up to 
100% in which case the entire screen (all pixels) will 
be ray traced and the present invention wilt not be 
used. As noted above, approximately 25% has been 
determined to give good resolution and a vastly 
increased rate of imaging. Step 3 determines the con- 
figuration of ray traced pixels which will be turned over 
to the display adapter 16 for filling. That is, the con- 
figuration of the ray traced pixels as a square, circle, 
rectangle, trapezoid or other polygon will be deter- 
mined, thus allowing the display adapter to fill the area 
(pixels) lying between these polygonally configured 
ray traced pixels. 

The actual ray tracing computations are then per- 
formed by CPU 18 of the computer graphics system 
(step 4). At step 5, the colour values for these ray 
traced pixels and their configuration is then transmit- 
ted to the display adapter 16 from CPU 18 and via bus 
28. Also at step 5, shading processor 12 fills (colour 
interpolates) the pixels lying between these ray traced 
pixels and displays the ray traced and colour interpo- 
lated pixels as a filled polygon, on display 10. Step 6 
determines whether the percentage of pixels to be ray 
traced (determined at step 2) have actually been ray 
traced. If so, then the method of the present invention 
proceeds to step 7 and ends. However, if there are 
pixels remaining to be ray traced, then the method 
returns to step 4 where additional ray tracing is per- 
formed. Also, it should be noted that step 7 ends the 
process of the present invention, but the CPU 18 may 
continue ray tracing, and colour image the previously 
colour interpolated pixels. 

Figure 3 shows another configuration of pixels on 
display 10. It should be noted that dots represent ray 
traced pixels and X's represent colour interpolated 
pixels. To invoke the configuration of Fig. 3, CPU 18 
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would ray trace every other pixel along the first line 
(rowO) of display 10, i.e. pixels 0, 2, 4,6, 8, 10, 12.... 
Next, CPU 18 will ray trace pixels 0,2 of row 2 such 
that a polygon (in this case a square) is bounded by 
pixels 0,2 of row 0 and pixels 0,2 of row 2. This poly- 
gon is then transmitted to shading processor 1 2 which 
colour interpolates values for pixel 1 of row 0, pixels 
0,1 .2 of row 1 and pixel 1 of row 2. This block of pixels 
is then displayed on CRT 10. 

Simultaneously to display of the first block of 
pixels, CPU 18 colour images pixel of 4 row 2, which 
forms another square, since the remaining three 
bounding pixels have previously been ray traced. This 
second block is then colour interpolated by shading 
processor 1 2 and values for pixel 3 of row 0, pixels 3,4 
of row 1 and pixel 3 of row 2 are found. Therefore, it 
can be seen how by ray tracing one additional pixel, 
such as pixel 4 of row 2, four pixels may then be colour 
interpolated. Further, it ran be seen how imaging pixel 
6 of row 2 would allow another four pixels to be colour 
interpolated and so forth. In this manner, i.e. using 
processor 12 as a hardware assist to colour imaging, 
the thousands of pixels present on a typical CRT can 
be colour imaged and colour interpolated to display a 
complex scene which would otherwise take a matter 
of hours or even days to display. 



Claims 



displaying the colour imaged pixels and 
the colour interpolated pixels included within said 
chosen configuration area ; and 

returning to said step of computing colour 
imaged values for another configuration area until 
the determined number of pixels have been dis- 
played. 

A method as claimed in any preceding claim, 
wherein said step of computing colour imaged 
values comprises the step of performing ray trac- 
ing operations. 

A system for displaying a colour image by inter- 
polating pixel colour values, comprising : 

interface means (22) responsive to user 
input to determine a number of pixels to be colour 
imaged ; 

a processing unit (18) to compute colour 
imaged values for each of said pixels included 
within the determined number ; and 

interpolation means (12) for using the col- 
our imaged values to interpolate colour values for 
each pixel not included within the determined 
number. 

A system as claimed in Claim 6 in which said 
interpolation means (12) is a processor adapted 
to perform colour shading. 



1. A method of displaying a colour image by inter- 
polating pixel colour values, said method com- 
prising the steps of : 

determining a number of pixels to be col- 
our imaged ; 

computing colour imaged values for each 
of said pixels included within the determined 
number ; and 

using the colour imaged values to interpo- 
late colour values for each pixel not included 
within the determined number. 

2. A method as claimed in Claim 1 for use in a sys- 
tem having a processor adapted to perform colour 
shading, in which said interpolation step is perfor- 
med by said processor. 

3. A method as claimed in Claim 1 or Claim 2 whe- 
rein said step of determining comprises the step 
of subdividing the image into configuration areas, 
each containing a portion of the determined num- 
ber of pixels to be colour imaged. 

4. A method as claimed in Claim 3 wherein the inter- 
polation step comprises the steps of : 

interpolating colour values for only those 
pixels that are contained within a chosen configu- 
ration area and have not been colour imaged ; 



8. A system as claimed in Claim 6 or Claim 7 whe- 
rein said interface means is adapted to subdivide 
35 the image into configuration areas, each contain- 

ing a portion of the determined number of pixels 
to be colour imaged. 

3. A system as claimed in Claim 8 wherein said 
40 interpolation means (12) is adapted to interpolate 

colour values for only those pixels that are con- 
tained within a chosen configuration area and 
have not been colour imaged, 

and wherein a display (10) displays the 
45 colour imaged pixels and the colour interpolated 

pixels included within said chosen configuration 
area. 

10. A system as claimed in any of Claims 6 to 9 whe- 
50 rein said processing unit (18) comprises means 

for performing ray tracing operations. 
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